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Image Processing Apparatus, Image Processing Method, and Computer 
Readable Recording Medium Recording Image Processing Program for 
Processing Image Obtained by Picking Up or Reading Original 

5 [0001] This application is based on application No. 2000-260961 filed in 
Japan, the content of which is hereby incorporated by reference. 
BACKGROUND OF THE INVENTION 
Field of the Invention 

[0002] The present invention relates to apparatus and method for image 
10 processing, and a computer readable recording medium in which an image 
processing program is recorded. More particularly, the present invention 
relates to an image processing apparatus, an image processing method, and 
a computer readable recording medium recording an image processing 
program for processing an image obtained by picking up or reading an 
15 original. 

Description of the Related Art 

[0003] Conventionally, a non-contact type image pickup device like a 
digital camera and a contact type input device like an image scanner have 
been used to convert an original having an image formed on a paper or the 

20 like to digital data. Once an original paper is folded, a fold line or a fold 
may remain thereon. When shooting such an original including the fold 
with a digital camera or reading it with an image scanner, the amount of 
reflected light would differ in the folded portion from the remaining portion 
due to surface irregularity at the folded portion. As a result, the image 

25 obtained by shooting would include noise at the folded portion, with its 

luminance value varied from that of the remaining portion. Similarly, the 
image read by the image scanner would include noise at the folded portion. 
[0004] Fig. 13 shows an original to be shot by a digital camera. The 
original 200 includes a region 201 having a photograph therein and a region 

30 203 having text therein. Original 200 has been folded and has a fold line 
205 remained thereon. 

[0005] Fig. 14 schematically shows an image obtained by shooting the 
original shown in Fig. 13 by a digital camera. Referring to Fig. 14, the 
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picked up image 250 includes a region 260 having original 200 expressed 
therein. Region 260 in turn includes a region 261 corresponding to photo 
region 201 of original 200, a region 263 corresponding to text region 203, and 
shade 265 corresponding to a portion around fold line 205 of original 200. 
Shade 265 appears because of the difference in the amount of reflected light 
in the folded portion from the remaining portion of original 200 due to the 
surface irregularity at fold line 205. 

[0006] When binarizing picked up image 205, an edge originally not 
existent on original 200 comes to appear, since the lightness is lower in 
shade 265 than in the remaining portion. This degrades accuracy in a 
character recognition process or the like following the binarizing process. 
In the case where image 250 is subjected to a process for extracting a 
rectangular region enclosing a photograph or the like therein, the accuracy 
of extraction would also be degraded. Fig. 15 shows a photo region that 
would be extracted when the photo region extracting process is conducted for 
the image shown in Fig. 14. Referring to Fig. 15, the portion of shade 265 is 
misjudged to have an attribute as a picture, so that the rectangular region 
267 has been extracted wrongly as the photo region. 

[0007] Besides the binarizing process, in a process of compressing an 
image picked up from an original graphic printed with a limited number of 
colors by indexing the colors, the rate of compression would be degraded due 
to a considerable increase in the number of colors at a portion within the 
image corresponding to the fold of the original. 

[0008] As a technique for correcting the shade appearing on an image 
obtained from pickup of an original due to a fold on the original, an image 
reader performing shading correction has been disclosed in Japanese Patent 
Laying-Open No. 1-165264. This image reader corrects a pixel value based 
on shading information obtained in advance from pickup of a white plate. 
[0009] Further, Japanese Patent Laying-Open No. 2-230870 discloses 
an image reader which detects a background luminance value of an original 
from an image obtained from pickup thereof, and ehminates the background 
based on the detected background luminance value. 

[0010] There is another method for correcting shading by additionally 



providing a sensor to detect height information of an original and estimating 
unevenness in illuminance from the detected height of the original, A 
sensor may be of the type that obtains the height information from a 
radiated result of spot light or slit light. 

[00 1 1] These conventional techniques however exhibit various problems. 
In the image reader disclosed in Japanese Patent Laying-Open No. 1-165264, 
the white plate is used. Since the white plate does not include a fold at a 
position corresponding to that of the fold on the original, correction of such a 
fold cannot be made only with the shading information obtained from the 
white plate. 

[00 12] With the image reader disclosed in Japanese Patent Laying- 
Open No. 2-230870, correction cannot be made for an image having almost 
no background region. 

[0013] Further, in order to detect a local change in height of the original 
due to a fold or the like with the method utilizing the height information 
obtained from a separate sensor, it is necessary to obtain the height 
information from a large number of positions. This increases the number of 
pieces of the height information, thereby decreasing the processing speed. 
In addition, to detect a minute change in height, an expensive sensor will be 
required or the device size will be increased. There also arises a need for 
detailed information about lighting conditions including position, direction 
and intensity of indoor or solar light. 
SUMMARY OF THE INVENTION 

[0014] The present invention is made to solve the above-described 
problems. An object of the present invention is to provide an apparatus 
and a method for image processing allowing removal of a noise appearing on 
a lightness component within an image. 

[00 15] Another object of the present invention is to provide an 
apparatus and a method for image processing allowing removal of a noise 
due to a fold on an original from an image. 

[00 16] A further object of the present invention is to provide a computer 
readable recording medium recording an image processing program to cause 
a computer to perform image processing allowing removal of a noise 
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appearing on a lightness component within an image. 

[0017] Yet another object of the present invention is to provide a 

computer readable recording medium recording an image processing 

program to cause a computer to perform image processing allowing removal 

of a noise due to a fold on an original from an image. 

[00 18] To achieve the above-described objects, the image processing 

apparatus according to an aspect of the present invention includes a 

receiving unit to receive an image, an edge detecting unit to detect an edge 

using a lightness component of the received image, a selecting unit to select 

a specific one of the detected edges, and a correcting unit to correct the 

lightness component of the selected specific edge. 

[0019] According to the present invention, it is possible to provide an 
image processing apparatus allowing removal of a noise appearing on the 
lightness component. 

[0020] Preferably, the image processing apparatus further includes a 
unit to extract an original included in the image, and the selecting unit 
selects an edge continuously extending from a first end to a second end of the 
extracted original. 

[002 1] According to the present invention, the edge uninterrupted from 
the first end to the second end of the extracted original is selected. If the 
original includes a fold, the fold will appear on a picked up image as a noise 
of shade. It is often the case that such a fold of the original extends from an 
end to another end thereof. Thus, by selecting the edge continuously 
extending from the first end to the second end of the original included in the 
image, the noise resulting from the fold of the original can be selected from 
the image. As a result, it is possible to provide an image processing 
apparatus allowing removal of a noise due to a fold of the original from the 
image. 

[0022] According to another aspect of the present invention, the 
computer readable recording medium recording an image processing 
program records the image processing program to cause a computer to 
perform the step of receiving an image picked up from an original, the step 
of detecting an edge using a lightness component of the received image, the 
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step of selecting a specific one of the detected edges, and the step of 
correcting the lightness component of the selected specific edge. 
[0023] According to the present invention, it is possible to provide a 
computer readable recording medium recording an image processing 
5 program to cause a computer to perform image processing that enables 
removal of a noise appearing on the lightness component. 
[0024] Preferably, the selecting step of the image processing program 
includes the step of selecting an edge continuously extending from a first 
end to a second end of the original included in the image. 

10 [0025] According to the present invention, it is possible to provide a 
computer readable recording medium recording an image processing 
program to cause a computer to perform image processing that enables 
removal of a noise due to a fold of the original from the image. 
[0026] According to another aspect of the present invention, the image 

15 processing method includes the step of receiving an image picked up from an 
original, the step of detecting an edge using a lightness component of the 
received image, the step of selecting a specific one of the detected edges, and 
the step of correcting the lightness component of the selected specific edge. 
[0027] According to the present invention, it is possible to provide an 

20 image processing method that enables removal of a noise appearing on the 
lightness component. 

[0028] Preferably, the selecting step of the image processing method 
includes the step of selecting an edge continuously extending from a first 
end to a second end of the original included in the image. 
25 [0029] According to the present invention, it is possible to provide an 
image processing method that enables removal of a noise resulting from a 
fold on the original from the image. 

[0030] According to yet another aspect of the present invention, the 
image processing apparatus includes an acquiring unit to acquire an image 
30 signal indicating an original image, an edge detecting unit to detect an edge 
in a lightness image primarily representing lightness of the original image 
as a lightness edge, a fold edge selecting unit to select any of the detected 
lightness edges that connects an edge corresponding to an end of an original 
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to another edge corresponding to another end of the original as a fold edge 
attributable to a fold of the original, and a processing unit to process the 
image signal in the portion corresponding to the fold edge to ehminate an 
effect of the fold of the original on the original image. 
5 [00311 According to the present invention, it is possible to provide an 

image processing apparatus allowing removal of a noise resulting from a fold 
of the original from the image. 

[0032] According to a further aspect of the present invention, the image 

processing apparatus includes an acquiring unit to acquire an image signal 
10 representing a color original image with three components, a color space 
P s converting unit to perform coordinate transformation of the image signal 

*@ such that the color original image is represented by a lightness component 

% primarily representing lightness and another component, and a correcting 

p unit to correct the lightness component in a portion of the color original 

^ 15 image that is detected as an edge portion in a lightness image including the 
y] lightness component and is undetected as an edge portion in a color 

difference image including the another component, so that the relevant 
S portion is undetected as the edge portion in the lightness image. 

[0033] According to the present invention, the edge appearing only on 
^ 20 the lightness component is selected as the noise. Thus, it is possible to 
H provide an image processing apparatus allowing accurate removal of a noise 

of the lightness component from the image. 

[0034] According to a still further aspect of the present invention, the 
image processing apparatus includes an acquiring unit to acquire an image 

25 signal representing an original image, an edge detecting unit to detect an 
edge in a lightness image primarily representing lightness of the original 
image as a lightness edge, and a correcting unit to correct a lightness 
component in a portion of the original image detected as the lightness edge 
when a difference in lightness between portions of the original image each at 

30 a prescribed distance from the relevant lightness edge on its respective sides 
is smaller than a prescribed threshold value, so that the relevant portion is 
undetected as the edge in the lightness image. 

[0035] According to the present invention, a boundary of two regions 
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whose difference in lightness is at least a threshold value is prevented from 
being wrongly selected as the noise. Thus, it is possible to provide an image 
processing apparatus allowing accurate removal of a noise in the lightness 
component from the image. 
5 [0036] According to yet another aspect of the present invention, the 

image processing apparatus includes an acquiring unit to acquire an image 
signal indicating an original image, an edge detecting unit to detect an edge 
in a lightness image primarily representing lightness of the original image 
as a lightness edge, and a correcting unit to correct a lightness component in 
10 a portion of the original image detected as the lightness edge when portions 
□ of the original image each at a prescribed distance from the relevant 

~[ lightness edge on its respective sides have the same attribute of image, so 

j| that the relevant portion is undetected as the edge in the lightness image. 

O [0037] According to the present invention, a boundary of two regions 

|J 15 having different attributes is prevented from being wrongly selected as the 
W noise. Thus, it is possible to provide an image processing apparatus 

^ allowing accurate removal of a noise of the lightness component from the 

m image. 

[0038] The foregoing and other objects, features, aspects and 
p 20 advantages of the present invention will become more apparent from the 
H following detailed description of the present invention when taken in 

conjunction with the accompanying drawings. 
BRIEF DESCRIPTION OF THE DRAWINGS 

[0039] Fig. 1 shows a digital camera in accordance with an embodiment 
25 of the present invention, which is picking up an image from an original. 

[0040] Fig. 2 is a perspective view of the digital camera in accordance 
with the embodiment seen from its front. 

[0041] Fig. 3 is a block diagram showing a circuit configuration of the 
digital camera in accordance with the embodiment. 
30 [0042] Fig. 4 illustrates how a portion of an image corresponding to a 
fold of an original is detected. 

[0043] Fig. 5 shows pixel values on a line (X = X0) in a lightness image. 
[0044] Fig. 6 illustrates correction carried out by a fold erasing unit of 
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the digital camera in accordance with the embodiment. 

[0045] Fig. 7 shows pixel values on the line (X = XO) in the lightness 

image after the correction. 

[0046] Fig. 8 is a flow chart illustrating a fold correcting process 
5 conducted by the digital camera in accordance with the embodiment. 
[0047] Fig. 9 is a flow chart illustrating a fold detecting process 
conducted by the digital camera in accordance with the embodiment. 
[0048] Fig. 10 shows an example of the image picked up by the digital 
camera in accordance with the embodiment. 
10 [0049] Fig. 1 1 shows pixel values on a line (X = X0) in the image shown 
in Fig. 10. 

[0050] Fig. 12 shows pixel values on a line intersecting the detected 
edge at right angles. 

[0051] Fig. 13 schematically shows an original to be shot by a digital 
15 camera. 

[0052] Fig. 14 schematically shows an image picked up by a digital 
camera from the original shown in Fig. 13. 

[0053] Fig. 15 shows a photo region extracted from the image shown in 
Fig. 14 as a result of a photo region extracting process. 

20 DESCRIPTION OF THE PREFERRED EMBODIMENTS 

[0054] Hereinafter, an image processing apparatus in accordance with 
an embodiment of the present invention will be described, taking a digital 
camera as an example. In the drawings, the same reference character 
denotes the same or corresponding portion, and description thereof will not 

25 be repeated where appropriate. 

[0055] Referring to Figs. 1 and 2, digital camera 1 includes a shooting 
button 3, a pickup lens unit 4 and a card slot 5. 

[0056] The image shot by digital camera 1 is stored as electronic data on 
a hard disc card (not shown) placed within digital camera 1. Here, the hard 
30 disc card is a recording medium of image data, for which the one in 

conformity with PCMCIA (Personal Computer Memory Card International 
Association) may be used, or a memory card or the like may be used. 
Alternatively, a mini disc (MD) or a digital video disc (DVD) may be 
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employed as the recording medium. Further, it is also possible to output 
the image data directly to a printer or the like via a SCSI (small computer 
system interface) cable, for example, instead of recording it on the recording 
medium. 

5 [0057] Digital camera 1 of the present embodiment can improve image 
quality when an original in the paper form, such as handout at a conference, 
catalog, magazine or research report, is recorded or output as electronic 
data. 

[0058] Fig. 3 is a block diagram showing a circuit configuration of 
10 digital camera 1 of the present embodiment. Referring to Fig. 3, digital 
camera 1 includes a central processing unit (hereinafter, "CPU") 100 
performing an overall control of digital camera 1, a charge coupled device 
(hereinafter, "CCD") 104 performing an image pickup, a display unit 106 
displaying an picked up image, a random access memory (hereinafter, 
15 "RAM") 108 temporarily storing the image from CCD 104, a color space 
converting unit 110 converting the picked up image to a prescribed color 
space, a pre-treatment unit 112 performing a prescribed pre-treatment on 
the converted image, a fold detecting unit 114 detecting a portion 
corresponding to a fold in the original from the image having undergone the 
20 pre-treatment at pre-treatment unit 112, a fold erasing unit 116 removing 
the detected fold, an output unit 118 outputting the image having the fold 
removed therefrom, a card memory unit 120 storing the image, and a read 
only memory (hereinafter, "ROM") 102 storing a program to be executed by 
CPU 100. 

25 [0059] Here, a reading device 122 may be connected to digital camera 1 
such that a program for the control of digital camera 1 can be read from 
CD-ROM 124, magneto-optical disc, digital video disc, flexible disk or the 
like. In this case, a program for causing CPU 100 to execute a fold 
correcting process, which will be described later, is recorded on a recording 

30 medium like CD-ROM 124 and is read by reading device 122, so that the fold 
correcting process is carried out. 

[0060] This fold correcting process may be carried out inside digital 
camera 1, or alternatively, it may be performed at another terminal such as 
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a personal computer or another camera connected to digital camera 1 by 
transferring the image data thereto. 

[006 1] An actual shooting process will now be described with reference 
to the block diagram shown in Fig. 3. In Fig. 3, thick arrows represent the 
5 flow of the image, and thin arrows represent the flow of the control data. 

When a user turns on the camera, a scene being taken by pickup lens unit 4 
is displayed on display unit 106 via CCD 104. 

[0062] Upon detection of pressing of shooting button 3, CPU 100 
designates integration to CCD 104. When the integration is completed, it 
10 dumps the CCD data to RAM 108, and causes display unit 106 to display the 
^ image in freeze-frame. 

3 [0063] The image from CCD 104 is stored in RAM 108. Here, the 

image stored in RAM 108 is a full-color image consisting of multi-valued 
O pixels of R (red), G (green) and B (blue) with 8 bits each. Once the image 

]f; 15 from CCD 104 is stored in RAM 108, it is subjected to respective processes in 
yj color space converting unit 110, pre-treatment unit 112, fold detecting unit 

^ 1 14 and fold erasing unit 1 16. The image having thus undergone the fold 

m correcting process is output from output unit 118 to card memory unit 120. 

i"U [0064] This card memory unit 120 may be a hard disc, or may be an 

n 20 external storage device or terminal. 

H [0065] Color space converting unit 110 converts the image stored in 

RAM 108 into color space having a lightness component. As the color space, 
YUV, L*a*b*, HSL or other color systems may be employed. In an 
environment for shooting an original, if the light radiated from a light 

25 source has a frequency component deflected to yellow or blue, for example, a 
new color system wherein a pixel value obtained by receiving the light 
radiated from the light source and reflected on a white plate is used as the 
lightness component without alteration may be established. 
[0066] Generally, a fluorescent light is used as the light source in the 

30 environment for shooting an original including text or the like. When the 
light source is the fluorescent light, it is considered that the component 
representing intensity of the light radiated from the fluorescent light is 
approximately equal to the Y component of the YUV color system. Thus, in 
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the present embodiment, color space converting unit 110 performs 
conversion to the color space of the YUV color system. 
[0067] Of the images converted by color space converting unit 1 10 into 
those expressed with lightness and other components, the one expressed 
5 only with the lightness component is called a lightness image, and the one 
expressed with the other components is called a color difference image. 
[0068] Pre-treatment unit 112 performs pre-treatment for edge 
detection on the lightness image generated by color space converting unit 
1 10. As the pre-treatment, sharpening process, contrast emphasizing 
10 process, minimum value filtering process, maximum value filtering process, 
smoothing process for noise removal, and image downsizing process are 

o 

.fi conducted in this order. The sharpening process is performed to emphasize 

u ij the edge of the image. The minimum value filtering process is performed to 

q convert a pixel value within a prescribed block to a minimum value for the 

^ 15 purpose of connecting the edge. The maximum value filtering process is 
conducted to replace a pixel value in a prescribed block with a maximum 
value so as to thin an edge that has become too thick or to remove a noise. 
.5 The image downsizing process is conducted to reduce the amount of data 

,y used for noise removal and edge detection. 

^20 [0069] It is preferred to change the order of these processes, the kinds or 
degrees of the processes as necessary, according to the entire configuration 
of the digital camera, fighting conditions at the time of shooting, kinds of 
subjects or the like, to increase the accuracy of the edge extraction. 
[0070] Pre-treatment unit 1 12 may conduct the pre-treatment for the 

25 image before being converted by color space converting unit 110. 

[0071] Fold detecting unit 114 detects a portion of the image having 
undergone the pre-treatment corresponding to the fold of the original. To 
this end, an absolute value of quadratic differential is first calculated for the 
image having undergone the pre-treatment to generate an edge image. The 

30 edge image thus represents the degrees of changes of pixel values. 
Accordingly, an edge in an image corresponds to a portion within the image 
where the pixel values show sudden changes. The edges corresponding to 
ends of the original are then detected from the generated edge image. 
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When the image includes the original therein, the edges corresponding to 
the ends of the original will be detected as an approximately rectangular 
shape. An edge connecting two points on these edges corresponding to the 
ends of the original is then detected as the edge corresponding to the fold of 
the original. This edge detecting process will be described later in detail. 
[0072] The edge detection may be conducted using any conventional 
techniques including those based on Hough transformation or edge tracing. 
[0073] To extract a region corresponding to the original, a method of 
detecting background luminance values from the image and using the 
detected results for the extraction, or a method utilizing edges may be 
employed. 

[0074] Fold erasing unit 1 16 conducts correction of the image stored in 
RAM 108 in its portion corresponding to the fold detected by fold detecting 
unit 1 14. This correction is performed only for the lightness image. In 
this correction, a pixel value of the lightness image is corrected according to 
its distance from a point on the portion corresponding to the detected fold. 
[0075] Fig. 4 illustrates how a portion of the lightness image 
corresponding to the fold of the original is extracted. The portion of the 
lightness image corresponding to the original 150 includes a plurality of 
edges. Among them, the one extending from an end to another end of 
original 150 is most likely to correspond to the fold of the original. In the 
example shown in Fig. 4, edges 151 and 152 are detected, since they are 
considered to correspond to the folds. Other edges are not detected as those 
corresponding to the folds, since they are considered as edges included in 
pictures or letters within the original. 

[0076] Fig. 5 shows pixel values on a line X = X0 in the lightness image 
shown in Fig. 4. Referring to Fig. 5, there is a sudden change of the pixel 
values at a position P0. Thus, the position P0 is detected as an edge. 
[0077] Fig. 6 illustrates a correcting method employed at the fold 
erasing unit of the digital camera of the present embodiment. In Fig. 6, the 
horizontal axis represents the position X of a pixel, and the vertical axis 
represents the pixel value in the lightness image. Referring to Fig. 6, a 
maximum correction amount Vis determined as half the amount of change 
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in pixel value between positions Q and R that are each at a minute distance 
d from the position P of the detected edge. The correction of the lightness 
image is performed for a region from a position Xb to a position Xa, where 
position Xb is a distance D apart from edge position P in the positive 
5 direction, and position Xa is distance D apart from edge position P in the 
negative direction. A pixel value Yl at a position XI in the negative 
direction from edge position P is set to Yl + V x (Xl - Xa) / D. A pixel value 
Y2 at a position X2 in the positive direction from edge position P is set to Y2 
- V x (Xb - X2) / D. Thus, the pixel values around edge position P in the 
10 lightness image are corrected and the change at position P is smoothed. 
[0078] Besides this correcting method, any other method may be 
employed which can smooth out the changes of the pixel values in the 
lightness image around edge position P. 

[0079] Fig. 7 shows pixel values on the line X = XO in the lightness 
15 image after correction. Referring to Fig. 7, the pixel values around edge 
y position PO have been corrected to smooth out the change at the position PO. 

;L [0080] The fold correcting process being conducted by the digital camera 

S of the present embodiment will now be described. Fig. 8 is a flow chart 

W showing the flow of the fold correcting process conducted by digital camera 1. 

jj^l 20 Referring to Fig. 8, in the fold correcting process, color space converting unit 
iM 1 10 first converts the image stored in RAM 108 into a lightness image and a 

color difference image in the color space of YUV color system (step Si). 
Thus, the lightness image composed only of the lightness component can be 
obtained from the full-color image shot by the camera. The following 
25 processes are conducted for the color space of the YUV color system. 

[008 1] Pre-treatment unit 1 12 conducts prescribed pre-treatment on the 
lightness image. As the pre-treatment, sharpening process, contrast 
emphasizing process, maximum value filtering process, minimum value 
filtering process, downsizing process and smoothing process are conducted 
30 in this order (step S2). 

[0082] Fold detecting unit 1 14 detects a portion of the lightness image 
corresponding to the fold of the original (step S3). A correction of the 
lightness image is then made for the detected portion corresponding to the 
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fold (step S4), and the process is terminated. 

[0083] Fig. 9 is a flow chart showing the flow of the fold detecting 
process conducted by digital camera 1 of the present embodiment. The fold 
detecting process corresponds to step S3 of Fig. 8. Referring to Fig. 9, in the 
fold detecting process, an edge image is first generated by calculating an 
absolute value of quadratic differential for each pixel of the lightness image 
having undergone the pre-treatment (step S 1 1). In the edge image, 
portions where the pixel values of the lightness image exhibit sudden 
changes are extracted as edges, which are differentiated from portions 
where the pixel values of the lightness image do not make sudden changes. 
[0084] Using the extracted edges, a region corresponding to the original 
included within the image, or the original region, is extracted (step S12). 
The extraction of this original region is performed by extracting edges 
forming a rectangle. Alternatively, the original region may be extracted by 
utilizing detected results of background luminance values. 
[0085] After the extraction of the original region, the edges included 
therein are examined, and any one continuously extending from an end to 
another end of the original is extracted therefrom (step S13). Specifically, 
the edges located in the vicinity of the ends of the original are first detected. 
It is then determined, for each pair of any two of the detected edges, whether 
there is an edge extending in the same direction and connecting the relevant 
pair of edges. In other words, it is determined whether there is an edge on 
a fine connecting the arbitrary two edges. If so, the relevant edge is 
extracted as the edge extending continuously from an end to another end of 
the original. 

[0086] If there is no edge connecting the arbitrary two edges, it is 
determined that the relevant pair of edges does not satisfy the requirement. 
Repeating this examination for every pair of edges located in the vicinity of 
the ends of the original, all the edges extending continuously from an end to 
another end of the original are detected. 

[0087] As the edge detecting method, the conventional methods based 

on the Hough transformation or edge tracing may be employed. 

[0088] In the next step S14, it is determined whether each detected edge 
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continuously extending from an end to another end of the original truly 
corresponds to the fold of the original. For this confirmation, a method . 
employing a color difference image, a method utilizing a background region 
of the image, and a method utilizing an attribute of the image may be 
employed by selectively using one of them or by combining two of them. 
[0089] 

[0090] Method Employing Color Difference Image 
[009 1] In this method, it is examined whether the edge continuously 
extending from an end to another end of the original extracted from the 
lightness image is also detected from the color difference image. If not, it is 
determined that the relevant edge corresponds to the fold of the original. 
This is because the noise due to the fold of the original would not change the 
color on the original, so that it should appear only on the lightness image. 
[0092] Method Utilizing Lightness of Background Region 
[0093] This method is advantageous for an original in which 
background regions different in lightness are located adjacent to each other. 
Fig. 10 shows an example of the image shot by digital camera 1 of the 
present embodiment. Referring to Fig. 10, the original included in the 
image has a title region and another region that are different in background 
color from each other. 

[0094] With this kind of original, a boundary between the title region 
and the other region will be extracted as the edge continuously extending 
from an end to another end of the original. However, if the correction as 
above is made for this edge, such an unnecessary correction will deteriorate 
the image quality. Thus, according to the present invention, background 
pixel values on opposite sides of the edge connecting an end to another end 
of the original are taken into consideration. In this example, they 
correspond to the background pixel values in the title region and in the other 
region each at a prescribed distance from the boundary. 
[0095] Fig. 1 1 shows pixel values on the line X = X0 within the lightness 
image shown in Fig. 10. Referring to Fig. 1 1, the background pixel values 
at positions at respective distance Yl, Y2 in opposite directions from the 
position P of the edge connecting two ends of the original are compared with 
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each other. In this example, the background pixel values in the title region 
and in the other region are different from each other. In such a case, it is 
determined that the extracted edge continuously connecting the two ends of 
the original does not correspond to the fold of the original. 
[0096] Fig. 12 shows pixel values on a line perpendicular to another 
detected edge. In the case where the background pixel values at positions 
at respective distance Yl, Y2 in opposite directions from the edge position P 
are equal to each other, it is determined that the relevant edge corresponds 
to the fold of the original. Thus, in this case, the correction is made for the 
relevant portion corresponding to the fold in step S4 of Fig. 8. 
[0097] Method Utilizing Attribute of Image 
[0098] In the method utilizing an attribute of an image, the 
determination as to whether the extracted edge truly corresponds to the fold 
of the original is made using the image attribute, instead of the lightness of 
the background regions used in the above-described method. Here, the 
image attribute of a region having a photograph included therein becomes a 
picture attribute, while a region having text therein exhibits a text 
attribute. 

[0099] In this method, in the case where two regions at the same 
distance from a detected edge in opposite directions perpendicular to the 
edge have the same image attribute, it is determined that the relevant edge 
corresponds to the fold of the original. If the two regions have the image 
attributes different from each other, it is determined that the relevant edge 
does not correspond to the fold of the original. Thus, in the case where a 
region including a photograph and exhibiting the picture attribute and a 
region including text and exhibiting the text attribute are located adjacent 
to each other, the edge of their boundary is prevented from being wrongly 
extracted as the edge corresponding to the fold of the original. As a result, 
an unnecessary correction and hence degradation of the image quality can 
be suppressed. 

[0 100] As explained above, according to the digital camera of the 
present embodiment, it is possible to remove a noise resulting from a fold of 
an original from an image picked up from the original. Since the correction 
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can be made using only the picked up image, it is unnecessary to 
additionally provide a reference plate for shading or a device for measuring 
a shape of the original. Thus, the configuration is simplified. 
[0101] Although the fold correcting process for the color image has been 
5 described representatively above, the digital camera of the present 

embodiment can also perform the similar process on a monochrome image. 
In this case, the process of determining continuity of the color difference 
component becomes unnecessary, since the monochrome image itself is the 
lightness image. 

10 [0102] For the purposes of speeding the processing and simplifying the 
p device configuration, it is also possible to eliminate the color space 

^ conversion even when using the color image. In this case, edges considered 

r p to correspond to the folds are extracted from respective plane images, from 

g which any one appearing in all the three plane images is extracted as the 

jU 15 edge corresponding to the fold. Further, only the G plane image of the color 
W image can be used, instead of the lightness image, to conduct the process 

Jl^ according to the present embodiment. 

IB [0 103] Although the fold correcting process of the present embodiment 

J!? has been described to be performed within the digital camera, it is also 

p 20 possible to read the fold correcting program recorded on a recording medium 
I** j n a personal computer for execution of the program therein. In this case, 

the image shot by the digital camera can be sent to the personal computer, 
where the noise corresponding to the fold can be removed from the shot 
image. 

25 [0 104] According to the digital camera of the present embodiment, an 
edge extending from an end to another end of the original is first selected, 
and it is then confirmed whether it corresponds to the fold of the original 
with "the method employing the color difference", "the method utilizing the 
background region of the image", or "the method utilizing the image 

30 attribute". Alternatively, it is also possible to confirm an edge 

corresponding to the fold of the original by first selecting edges with any of 
"the method employing the color difference", "the method utilizing the 
background region of the image", "the method utilizing the image attribute" 
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or a combination thereof, and then selecting among them the one extending 
from an end to another end of the original. For an image which does not 
include ends of the original therein, the edge corresponding to the fold of the 
original may be detected using any of "the method employing the color 
difference", "the method utilizing the background region of the image", "the 
method utilizing the image attribute", or a combination thereof. 
[0105] Further, for an image which includes an original under a limited 
condition, e.g., an original including only a text region and having its 
background color uniform through the original, it is possible to determine an 
edge extending from an end to another end of the original included within a 
generated edge image as the edge corresponding to the fold, without further 
processing. 

[0106] Although color space converting unit 106 has been provided in 
the present embodiment for converting the obtained image to the color space 
iy 15 having the lightness component, there are cases where such a unit is 
m unnecessary. An example thereof is the case where the image processing 

™ apparatus acquires as its input image an image represented by the color 

K space having the lightness component. Another example is the case where 

R, G and B values of each pixel are integrated to obtain a lightness value of 
p 20 the relevant pixel, and a lightness image is formed of those lightness values. 

In this case, the R, G and B values can be increased or decreased in scale in 
the portion extracted as an edge corresponding to the fold, so that the 
correction to eliminate a noise resulting from the fold of the original is 
realized. 

25 [0 107] Although the present invention has been described and 

illustrated in detail, it is clearly understood that the same is by way of 
illustration and example only and is not to be taken by way of limitation, the 
spirit and scope of the present invention being limited only by the terms of 
the appended claims. 
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